Add the short_name scope attribute #2702

dashpole · 2022-07-28T20:23:05Z

Changes

Add the short_name scope attribute. Alternatively, it could have been added as otel.short_name to distinguish it from non-otel attributes, or prometheus.namespace to make it explicitly associated with prefixing prometheus metrics.

@jmacd @tigrannajaryan @jsuereth

joaopgrassi

I believe it would make things more streamlined if we first tackle the addition of "scope" as a semantic convention concept separately from the Prometheus use-cases/changes. So, a PR only for adding the scope new yaml/md.

This (or a new PR if you decide to split) probably should fix #2682

semantic_conventions/scope/scope.yaml

dashpole · 2022-07-29T13:37:34Z

I believe it would make things more streamlined if first tackle the addition of "scope" as a semantic convention concept separately from the Prometheus use-cases/changes. So, a PR only for adding the scope new yaml/md.

Done

tigrannajaryan · 2022-07-29T14:31:42Z

@dashpole you said here that

I think the answer should be yes (namespaces are different), although we should try and avoid unnecessary collisions where possible to reduce confusion. I believe this is the same answer as the separation between resource vs trace/metric/log attributes.

If the Scope has a different namespace then why are we prefixing the attribute names by scope.? That seems unnecessary to me. It would be necessary if we decided that the Scope attribute names are in the same namespace as the Resource attributes and/or Trace or Metric attributes.

I am not sure what is the right approach but it seems like it needs to be one or another and then we need to consistent with that decision, which for now we don't seem to be.

specification/scope/README.md

semantic_conventions/scope/scope.yaml

specification/scope/README.md

dashpole · 2022-07-29T17:33:45Z

If the Scope has a different namespace then why are we prefixing the attribute names by scope.? That seems unnecessary to me. It would be necessary if we decided that the Scope attribute names are in the same namespace as the Resource attributes and/or Trace or Metric attributes.

My thinking was that some scope attributes may only apply to a particular definition of scope, such as module.path, or library.something... But I'm also having trouble coming up with examples, so maybe its a stretch. I'm happy to change it to short_name if you think it makes more sense

tigrannajaryan · 2022-07-29T17:54:18Z

I'm happy to change it to short_name if you think it makes more sense

It does look a bit weird. Virtually every other attribute we have in semantic conventions for Resources or Spans or Metrics has some sort of a prefix. We have never defined any attributes in the root namespace although I don't think the rules explicitly prohibit it.

However using a prefix scope. in a root namespace what is already asserted to be entirely allocated to the scopes looks weird too. Is there a way to come up with a different meaningful prefix that in the future may contain more than just the short_name?

On the other hand the benefit of using scope. prefix is that it mitigates the problem described in #2535. If scope. is used as a prefix then in practice it is never going to clash with Resource or Span attribute names, so how we define the precedence will almost not matter.

Sorry, I am probably not helping much. I probably need to make up my mind first :-)

joaopgrassi · 2022-08-01T14:39:55Z

It does look a bit weird. Virtually every other attribute we have in semantic conventions for Resources or Spans or Metrics has some sort of a prefix. We have never defined any attributes in the root namespace although I don't think the rules explicitly prohibit it.

I find strange that we have otel.scope.name and then scope.short_name. Are we completely opposed to have it under otel.scope.short_name? When I read the linked Attribute naming page above:

Attribute names that start with otel. are reserved to be defined by OpenTelemetry specification. These are typically used to express OpenTelemetry concepts in formats that don't have a corresponding concept.

It seems short_name falls into that well, no? It's like a sibling to name and version in regards of how important it is. Even so, if short_name is going to be used for #2703. It carries more meaning than an arbitrary attribute on a scope.

Then, other scope attributes don't need any prefix, and can exist by themselves, we just happen to "list" them under the scope semantic conventions bcs it's where they should be applied. And this ties nicely with the next thing:

On the other hand the benefit of using scope. prefix is that it mitigates the problem described in #2535

I would think it's better to not have a mixed approach when it comes to solving the question of how we "flatten" attributes. Like, either all signals have their attributes prefixed or none have. Having a prefix for scope and not for resource or span sounds weird to me, inconsistent.

semantic_conventions/scope/scope.yaml

dashpole · 2022-08-02T15:39:43Z

One topic that came up during today's spec call is the idea of making this attribute's behavior consistent across exporters. For example, prometheus adds it as a prefix to metrics, but why shouldn't other exporters do the same? Perhaps other non-OTLP exporters should also add this as a prefix. Maybe something like name_prefix would be clearer that the intention is for this to be a name prefix

jack-berg · 2022-08-02T19:31:03Z

specification/scope/README.md

+<!-- semconv scope -->
+| Attribute  | Type | Description  | Examples  | Requirement Level |
+|---|---|---|---|---|
+| `scope.short_name` | string | The short name for the instrumentation scope, which should generally be less than 15 characters, and generally consist of a single word. It is not required to be globally unique, but should be unique enough to make it very unlikely to collide with other short names.  An example use is as the [namespace](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#metric-naming-and-namespaces) (prefix) for OpenMetrics metric names. | `mylibrary`; `otelhttp` | Recommended |


Not a blocking comment, but it seems likely that scope.short_name will make its way into scope attributes for traces and logs as well. I can't think of any reason why this would be a problem - just want to confirm that everyone understands scope.short_name to be a signal agnostic attribute.

Is this attribute "special" in some way, compared with others? In the companion #2703 an example is given with two scope attributes, scope.short_name and library_mascot.

If I receive an OTLP export (for any signal) with two scopes that contain otherwise identically-named logs/spans/metrics, then:

the scope.short_name is implicit qualifier for the named logs/spans/metrics, making them distinct and unrelated; metrics belonging to the two scopes cannot internally break the single-writer rule (despite sharing same attributes).

the library_mascot is just like a resource attribute, has no special properties; metrics belonging to the two scopes would break the single-writer rule, if they were to share the same attributes.

Not sure if my interpretation is correct or not, would like this section to clarify.

just like a resource attribute, has no special properties; metrics belonging to the two scopes would break the single-writer rule, if they were to share the same attributes.

This doesn't seem correct for resource attributes. Reporting CPU usage (assume the metric has no attributes) is possible for two different resources (e.g. two different containers) without breaking the single-writer rule, right?

My take is that:

Fundamentally, the scope (full) name+ version is the implicit qualifier which implies that metrics within it distinct and unrelated. I believe this is currently the status quo, but this may be the question you are asking about.

All scope attributes are descriptive (not identifying) of the scope name + version

short_name is a shortened identifier which is likely, but not guaranteed, to uniquely identify the scope name + version within a program.

OK, thank you. I prefer your interpretation.
In data-model.md, there's a paragraph that can use a slight extension:

Metric streams are grouped into individual `Metric` objects, identified by: - The originating `Resource` attributes - The instrumentation `Scope` (e.g., instrumentation library name, version) - The metric stream's `name`

Maybe change "(e.g., instrumentation library name, version)" to "(i.e., instrumentation scope name, version, and attributes)".

I was actually thinking that scope attributes would be non-identifying: "All scope attributes are descriptive (not identifying) of the scope name + version", as that would reduce the burden on non-otel exporters to REQUIRE them to be attached to all telemetry. Are there use-cases where we want attributes to be identifying?

I don't think we want to have them, since we anyway don't use them to construct different meters.

I don't think we want to have them, since we anyway don't use them to construct different meters.

I expect varying scope attributes will produce different meters, seems to be the intention behind #2579, specifically https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md#get-a-meter

specifically "[since 1.13.0] attributes (optional): Specifies the instrumentation scope attributes to associate with emitted telemetry." and "Implementations MUST return different Meter instances when called repeatedly with different values of parameters."

dashpole · 2022-08-02T19:52:33Z

I've opted to update the PR to use the prefix-less short_name instead of scope.short_name based on discussions above.

tigrannajaryan · 2022-08-03T14:58:33Z

Regarding using scope. or otel. as a prefix: even if we use it for some attributes (like scope.short_name) we must still allow some other attributes that don't have the prefix. An example use case is the Events and Logs API, where we propose to have a Scope attribute named event.domain. I believe event.domain is a good name for an attribute and there is no need to prefix it with scope. or otel..

So, at the very least, IMO, we cannot have a hard rule that says scope. or otel. must be always a prefix for Scope attributes.

joaopgrassi · 2022-08-05T14:02:40Z

I've opted to update the PR to use the prefix-less short_name instead of scope.short_name based on discussions above.

@dashpole does the tooling works in this case, to auto generate the table? (without a prefix)

dashpole · 2022-08-05T15:37:33Z

@dashpole does the tooling works in this case, to auto generate the table? (without a prefix)

I had to add one commit to the scope build-tools PR to allow an empty prefix, but otherwise it works.

This unblocks #2702 by making the changes introduced in open-telemetry/build-tools#114 available for use. It also allows for more semantic conventions for scope attributes to be defined in the future (#2682). See https://github.com/open-telemetry/build-tools/releases/tag/v0.14.0.

Aneurysm9 · 2022-08-16T15:47:39Z

As discussed in the spec SIG today I wonder whether it would be beneficial to define a requirement that scope attributes be prefixed with otel.scope when processed by non-OTel components. This would align with otel.scope.name and otel.scope.version and provide a clear indication that an attribute was a scope attribute and not an attribute on a signal item.

tigrannajaryan · 2022-08-16T17:39:35Z

As discussed in the spec SIG today I wonder whether it would be beneficial to define a requirement that scope attributes be prefixed with otel.scope when processed by non-OTel components. This would align with otel.scope.name and otel.scope.version and provide a clear indication that an attribute was a scope attribute and not an attribute on a signal item.

This results in a desirable outcome for short_name however I am not so sure it is desirable for other attributes. For example there is an ongoing discussion to use event.domain scope attribute. Prepending it with otel.scope will result in otel.scope.event.domain which is unnecessarily long and and harder to read. I think keeping event.domain as is would be better - it is shorter and meets the uniqueness requirements since we can make event. a semantic convention namespace.

carlosalberto · 2022-08-17T14:47:01Z

Is it to bad to suggest that only root-level attributes get a suffix (short_name becomes scope.short_name or otel.scope.short_name), as they are de facto considered meta attributes?

carlosalberto · 2022-08-17T14:47:41Z

Approving, but as mentioned in the Spec call too, let's allow some days for some days to iron out final details.

jmacd · 2022-08-17T19:57:37Z

(I could not attend the Spec SIG yesterday and the recording is not yet available.)

From the discussion above, looks like we're still debating what to call this attribute and whether it should have "otel.scope" as a prefix.

@dashpole asked two weeks ago:

Maybe something like name_prefix would be clearer that the intention is for this to be a name prefix

I suggest that namespace best describes the thing we're talking about. The Prometheus exporter would be specified to construct metric names by stitching the namespace, _, and the metric name together. This namespace concept that we are describing exactly matches OpenMetrics, which is nice.

I think otel.scope. should remain the suggested prefix for instrumentation scope name and version concepts, because they are OpenTelemetry concepts. I would not automatically apply a otel.scope. prefix to any of the scope attributes. (Tangent: My guess is that a standard prefix is intended to make it easier to flatten the attributes into a list. Question: why flatten in the first place, when OpenMetrics specifically describes the use of join??)

dashpole · 2022-08-18T03:28:46Z

Why flatten in the first place, when OpenMetrics specifically describes the use of join?

I don't think we should flatten for Prometheus/OpenMetrics. I've proposed a join-able info metric in #2703. I think the guidance is for other non-OTLP exporters.

A scope is already somewhat of a namespace, and i'm hoping to introduce something thats closely tied to the scope name to avoid creating a new concept. I also worry an un-prefixed namespace would collide with metric/span/logRecord attributes often.

tigrannajaryan · 2022-08-18T14:59:22Z

I suggest that namespace best describes the thing we're talking about. The Prometheus exporter would be specified to construct metric names by stitching the namespace, _, and the metric name together. This namespace concept that we are describing exactly matches OpenMetrics, which is nice.

Is this Scope namespace concept only useful for metrics or we can generalize it and use uniformly for all signals?

github-actions · 2022-08-26T04:02:58Z

This PR was marked stale due to lack of activity. It will be closed in 7 days.

dashpole · 2022-08-31T18:50:09Z

Based on #2703 (comment), this isn't required anymore.

This unblocks open-telemetry#2702 by making the changes introduced in open-telemetry/build-tools#114 available for use. It also allows for more semantic conventions for scope attributes to be defined in the future (open-telemetry#2682). See https://github.com/open-telemetry/build-tools/releases/tag/v0.14.0.

dashpole requested review from a team July 28, 2022 20:23

github-actions bot assigned jmacd Jul 28, 2022

arminru added area:semantic-conventions Related to semantic conventions spec:metrics Related to the specification/metrics directory labels Jul 29, 2022

joaopgrassi reviewed Jul 29, 2022

View reviewed changes

semantic_conventions/scope/scope.yaml Outdated Show resolved Hide resolved

dashpole force-pushed the short_name_scope_attribute branch from a788e5e to 3c80ea8 Compare July 29, 2022 13:31

dashpole changed the title ~~Use the scope.short_name scope attribute to support instrumentation scope in Prometheus~~ Add the scope.short_name scope attribute Jul 29, 2022

This was referenced Jul 29, 2022

Add scope semantic conventions as a type open-telemetry/build-tools#114

Merged

Add Instrumentation Scope and Version as labels in Prometheus #2703

Merged

joaopgrassi reviewed Jul 29, 2022

View reviewed changes

specification/scope/README.md Outdated Show resolved Hide resolved

semantic_conventions/scope/scope.yaml Outdated Show resolved Hide resolved

specification/scope/README.md Show resolved Hide resolved

tigrannajaryan reviewed Jul 29, 2022

View reviewed changes

specification/scope/README.md Outdated Show resolved Hide resolved

dashpole force-pushed the short_name_scope_attribute branch from 5f7c784 to 4ee1ff5 Compare July 29, 2022 19:36

jack-berg reviewed Aug 1, 2022

View reviewed changes

semantic_conventions/scope/scope.yaml Outdated Show resolved Hide resolved

dashpole force-pushed the short_name_scope_attribute branch from d639496 to 5b37f17 Compare August 2, 2022 19:06

dashpole changed the title ~~Add the scope.short_name scope attribute~~ Add the short_name scope attribute Aug 2, 2022

jack-berg approved these changes Aug 2, 2022

View reviewed changes

dashpole mentioned this pull request Aug 10, 2022

Collector metrics names reported with OpenCensus and OpenTelemetry are inconsistent open-telemetry/opentelemetry-collector#5882

Closed

arminru mentioned this pull request Aug 12, 2022

Upgrade semantic convention tool to v0.14.0 #2725

Merged

dashpole added 4 commits August 16, 2022 14:23

add scope.short_name scope attribute

9a86565

add link to instrumentation scope

3919f66

update description of short_name

7bda4a3

update wording, update short_name example, and remove prefix

61038e4

dashpole force-pushed the short_name_scope_attribute branch from 7808057 to 61038e4 Compare August 16, 2022 14:24

use empty-string as the prefix

b42d588

dashpole force-pushed the short_name_scope_attribute branch from 740c4a0 to b42d588 Compare August 16, 2022 14:35

regenerate tables

e45d687

dashpole force-pushed the short_name_scope_attribute branch from ee634c0 to e45d687 Compare August 16, 2022 14:55

carlosalberto approved these changes Aug 17, 2022

View reviewed changes

jack-berg mentioned this pull request Aug 17, 2022

Add scope attributes open-telemetry/opentelemetry-java#4695

Open

7 tasks

github-actions bot added the Stale label Aug 26, 2022

dashpole closed this Aug 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the short_name scope attribute #2702

Add the short_name scope attribute #2702

dashpole commented Jul 28, 2022 •

edited

Loading

joaopgrassi left a comment •

edited

Loading

dashpole commented Jul 29, 2022

tigrannajaryan commented Jul 29, 2022

dashpole commented Jul 29, 2022

tigrannajaryan commented Jul 29, 2022

joaopgrassi commented Aug 1, 2022

dashpole commented Aug 2, 2022

jack-berg Aug 2, 2022

jmacd Aug 9, 2022

dashpole Aug 9, 2022 •

edited

Loading

jmacd Aug 9, 2022

dashpole Aug 9, 2022

bogdandrutu Aug 10, 2022

jmacd Aug 17, 2022

dashpole commented Aug 2, 2022

tigrannajaryan commented Aug 3, 2022

joaopgrassi commented Aug 5, 2022

dashpole commented Aug 5, 2022

Aneurysm9 commented Aug 16, 2022

tigrannajaryan commented Aug 16, 2022 •

edited

Loading

carlosalberto commented Aug 17, 2022

carlosalberto commented Aug 17, 2022

jmacd commented Aug 17, 2022

dashpole commented Aug 18, 2022

tigrannajaryan commented Aug 18, 2022

github-actions bot commented Aug 26, 2022

dashpole commented Aug 31, 2022

Add the short_name scope attribute #2702

Add the short_name scope attribute #2702

Conversation

dashpole commented Jul 28, 2022 • edited Loading

Changes

joaopgrassi left a comment • edited Loading

Choose a reason for hiding this comment

dashpole commented Jul 29, 2022

tigrannajaryan commented Jul 29, 2022

dashpole commented Jul 29, 2022

tigrannajaryan commented Jul 29, 2022

joaopgrassi commented Aug 1, 2022

dashpole commented Aug 2, 2022

jack-berg Aug 2, 2022

Choose a reason for hiding this comment

jmacd Aug 9, 2022

Choose a reason for hiding this comment

dashpole Aug 9, 2022 • edited Loading

Choose a reason for hiding this comment

jmacd Aug 9, 2022

Choose a reason for hiding this comment

dashpole Aug 9, 2022

Choose a reason for hiding this comment

bogdandrutu Aug 10, 2022

Choose a reason for hiding this comment

jmacd Aug 17, 2022

Choose a reason for hiding this comment

dashpole commented Aug 2, 2022

tigrannajaryan commented Aug 3, 2022

joaopgrassi commented Aug 5, 2022

dashpole commented Aug 5, 2022

Aneurysm9 commented Aug 16, 2022

tigrannajaryan commented Aug 16, 2022 • edited Loading

carlosalberto commented Aug 17, 2022

carlosalberto commented Aug 17, 2022

jmacd commented Aug 17, 2022

dashpole commented Aug 18, 2022

tigrannajaryan commented Aug 18, 2022

github-actions bot commented Aug 26, 2022

dashpole commented Aug 31, 2022

dashpole commented Jul 28, 2022 •

edited

Loading

joaopgrassi left a comment •

edited

Loading

dashpole Aug 9, 2022 •

edited

Loading

tigrannajaryan commented Aug 16, 2022 •

edited

Loading